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Abstract 

As interest increases in mindfulness in education programs for youth, there is a need to develop reliable measures of the quality of 
program implementation. This paper describes the development and psychometric properties of a measure that can be used to 
assess and monitor quality of implementation of mindfulness programs/curricula in typical classroom or out-of-school settings. 
The Teaching Mindfulness in Education Observation Scale (TMEOS) is a 28-item instrument that integrates qualitative and 
quantitative aspects of mindfulness instruction. Items focus on procedural adherence as well as aspects of implementation that 
reflect embodiment during instructional delivery (e.g., alignment with the attitudinal foundations of mindfulness). Reliability and 
validity data were examined and indicated that the four major scoring domains and key features showed adequate inter-rater 
reliability and internal consistency. We conclude that observational assessment of multiple dimension of implementation quality, 
including adherence and process-oriented aspects of implementation such as embodiment, can be reliably used to assess imple- 
mentation quality of mindfulness programs in education settings. However, adequate preparation and training are critical. 
Implications and directions for future research are discussed. 


Keywords Fidelity - Implementation - Training - Professional development - Coaching - Mindfulness - Schools - Education 


Mindfulness-based interventions (MBIs) have grown increas- 
ingly popular around the world over the past few decades, 
generating substantial interest in applications for children and 
youth (Frank et al. 2013). Recent meta-analyses have sug- 
gested that mindfulness-based interventions in education 
(MBI-ED) hold promise (Burke 2010; Zoogman et al. 2015). 
However, questions about dosage, frequency, feasibility, ac- 
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ceptability, outcomes, and developmental appropriateness of 
mindfulness programs remain (Greenberg and Harris 2012; 
Maynard et al. 2017). Much more research needs to be done 
to strengthen our understanding of mindfulness interventions in 
education because programs in current use often suffer from 
lack of empirical support (Semple et al. 2017). Providing a 
reliable means of assessing fidelity of program implementation 
could strengthen this body of research, support instructor train- 
ing, and contribute to program improvement (Durlak 2016; 
Gould et al. 2014). 

Assessing implementation fidelity requires attention to two 
major domains: program adherence (e.g., what was done) and 
quality or process of implementation (e.g., how it was done). 
Such assessment requires a clear delineation of the core struc- 
tural and process-oriented components of the program. The 
quality or process-oriented components are of particular impor- 
tance in MBIs. Presumably, it is the mindfulness instructor’s 
embodiment of mindfulness in the way s/he conducts the pro- 
gram and interacts with participants that provides the essential 
model of mindfulness in action (see McCown et al. 2011, 2016). 


a Springer 


Despite the notable exception of a measure of mindfulness 
teaching quality developed in the UK (Crane et al. 2012), few 
studies have assessed program implementation in even the most 
rudimentary way. In a review of three recent literature reviews 
of research on the effectiveness of mindfulness-based interven- 
tions (Chiesa and Serretti 2009; Grossman et al. 2004; Virgili 
2013), Harrison (2014) found that very few of the studies 
reviewed even considered how the amount of mindfulness 
training an individual underwent (e.g., “dose”’) related to out- 
comes. Specifically, only between 11 and 29% of the highest 
quality studies in the field on mindfulness training (those that 
met rigorous criteria of validity) examined the question of 
whether or not program dose was associated with the outcomes 
of training (Harrison 2014). This is also true in education, 
where existing research on mindfulness intervention has not 
addressed implementation quality with respect to MBIs for ed- 
ucators, children, and youth, even at the basic level of dose (see 
Harrison 2014). At present, there are no published and available 
assessment measures to assess the quality of implementation for 
MBI-ED programs. Gould et al. (2014) reviewed 48 yoga and 
mindfulness reports with youth and found that fewer than 20% 
assessed any aspect of implementation fidelity beyond partici- 
pant dosage, and only 4% of studies observed and rated pro- 
gram quality. When included in these studies, program quality 
was assessed using participant feedback about instructor perfor- 
mance or instructor self-report, not the observed quality of the 
program implementation. 

Being mindful, or intentional and present-focused with com- 
passionate awareness, involves both internal and interpersonal 
processes. When mindful, one is aware of bodily sensations and 
habitual, well-practiced patterns of thoughts and emotions as 
they are occurring. Because present experience is approached 
with curiosity, acceptance, and nonjudgment, its transitory na- 
ture becomes more apparent. Habitual ways of acting and 
reacting, both in relation to ourselves and to others, become 
less automatic and limiting. An increased sense of efficacy 
and the potential for greater discernment may emerge with 
more freedom for choosing how to respond. Mindful awareness 
shifts our relationship to everyday life and helps us live more 
fully in the present. As such, in teaching mindfulness practices 
to others, core components of the program not only include 
adhering to the program model, but embodying the attitudinal 
foundations of mindfulness as well (Kabat-Zinn 1990). 

These attitudes and correspondent behaviors represent what 
many call “embodiment” (Brown et al. 2016; Roeser 2016), and, 
according to experienced mindfulness instructors, are the fruit of 
deliberate practice (Crane et al. 2012; Cullen 2011; Jennings 
2015; McCown et al. 2011, 2016). Crane et al. (2016) identify 
“embodiment of foundational mindfulness qualities” as one of 
the four distinguishing characteristics of MBI instructors. Other 
characteristics include knowledge of and competency in teaching 
MBIs, commitment to mindfulness practice, and participation 
with students in a process of inquiry during the teaching process. 
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In the more fully developed field of MBIs for adults, ex- 
emplary teacher training and certification programs have been 
developed for well-established interventions like mindfulness- 
based stress reduction (MBSR; Center for Mindfulness in 
Medicine, Health Care and Society http://www.umassmed. 
edu/cfm/) and mindfulness-based cognitive therapy (MBCT; 
Center for Mindfulness in Center for Mindfulness Research 
and Practice https://www.bangor.ac.uk/mindfulness/). These 
training programs include best practice guidelines, 
supervision, and assessment rubrics, though the use of these 
assessment rubrics remains rare in existing research studies. 

As interest in evidence-based mindfulness programs for 
youth increases and evidence accumulates for its effectiveness, 
it will be essential that teachers in schools are trained to a high- 
quality standard and deliver programs with integrity in order to 
obtain the desired outcomes, as is the case in adult programs. In 
addition to the traditional implementation measures of adher- 
ence (delivery of program components), instructor embodi- 
ment of qualities of mindfulness is seen as a critical component 
for engaging students in participatory learning in their own 
lives (Crane et al. 2012; Gould et al. 2014; Kabat-Zinn et al. 
2011; Roeser 2016). 

For example, it is consistent with the nature of mindfulness 
to be “in the moment” and to respond flexibly to students at any 
given point in time, rather than adhering rigidly to a manual 
(Kabat-Zinn et al. 2011). Indeed, the ability of the instructor to 
be flexible in responding to the “present moment” of what is 
unfolding is a core component for MBIs. This issue of being 
embodied and flexible while at the same time covering the core 
components of the specified program presents intricate 
challenges for both instructors and the assessment of 
implementation. Dobkin et al. (2014) emphasize the impor- 
tance of maintaining the “heart of the mindfulness 
intervention” while adapting it for specific populations as well 
as the need to join with participants in skillfully introducing 
mindfulness into specific contexts. 

In addition to population and contextual issues, if mindful- 
ness is to be integrated in educational settings, it will likely 
require that universal, school-based programs will be taught 
by classroom teachers. Training classroom teachers to become 
mindfulness instructors represents a different set of challenges 
compared to training mindfulness instructors for adult MBIs. In 
educational settings, teachers are not clinicians, and programs 
tend to be simpler and somewhat more directive, befitting 
younger audiences. In most clinical settings, participants have 
volunteered and thus have some personal motivation to learn 
and practice mindfulness skills, whereas in classrooms, children 
may not be motivated or interested in such skills. Thus, just as 
context demands developmentally appropriate programs, con- 
text also demands non-clinical, alternative ways of training and 
supporting teachers to deliver MBIs for students. An influential 
meta-analysis of universal SEL programs (Durlak et al. 2011) 
showed that school personnel were more effective teachers of 
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SEL skills compared to outside experts, presumably because of 
their established relationships with students and ability to gen- 
eralize the new skills across the classroom day. 

In order to assure sustainability of mindfulness-based inter- 
ventions, it may be necessary for school personnel to be the 
primary instructors of mindfulness in these settings. If this is 
SO, it raises a number of questions such as (1) what cultivation 
of their own mindfulness is needed to lead mindfulness with 
students? (2) what kind of training, supervision, and support is 
necessary to move teachers toward mindful embodiment, 
which is unlike most standard educational pedagogy? (3) what 
kinds of translation can be made to educators to communicate 
and support the foundational attitudes that appear to be central 
to embodying mindfulness? 

One existing assessment measure with some research sup- 
port, the mindfulness-based interventions: teaching assessment 
criteria (MBI:TAC; Crane et al. 2013), offers a springboard for 
the current measure development and adaptation to classroom 
teachers and other educators. The MBI:TAC was developed to 
train and assess clinicians utilizing mindfulness-based cognitive 
therapy (MBCT), an 8-week program intended to prevent de- 
pressive relapse. The MBI:TAC assesses multiple domains in- 
cluding (1) coverage, pacing, and organization; (2) relational 
skills; (3) embodiment of mindfulness; (4) guiding mindfulness 
practices; (5) conveying course themes through interactive in- 
quiry and didactic teaching; and (6) holding the group learning 
environment. Assessments may be made on a single session or 
the whole 8-week program. The MBI:TAC is used for instructor 
training purposes, and raters “need to be very experienced 
mindfulness-based instructors in the particular program 
(MBCT) they are assessing (R. Crane, personal communication, 
Jan. 11, 2017).” 

Embodiment of mindfulness by the instructor is a critical 
domain in the MBI:TAC, one of the core features of mindful 
teaching (Crane et al. 2016) and quite possibly one of the most 
challenging to translate into observable behavior. Brown et al. 
(2016) argue that embodiment is not an “‘add-on” set of be- 
haviors but an authentic reflection of the teacher and the mod- 
el of presence through which students learn to be mindful 
themselves. The key quality we are discussing here is not 
simple “embodiment” but the mindfulness instructor’s em- 
bodied presence, what Rodgers and Raider-Roth (2006) de- 
fine as “A state of alert awareness, receptivity and connected- 
ness to the mental, emotional, and physical workings of both 
the individual and the group in the context of their learning 
environments and the ability to respond with a considered and 
compassionate next step” (p. 266). 

Kabat-Zinn (1990) identified seven foundational attitudes 
of mindfulness that constitute the cognitive, affective, and 
behavioral aspects of “embodiment.” These attitudinal foun- 
dations include non-judgment, patience, acceptance, begin- 
ner’s mind, trust, non-striving, and letting go. Kabat-Zinn used 
the term “attitudes” instead of “behaviors” to communicate a 


manner of approaching all experience, including classroom 
experience, in a way that goes beyond simply describing be- 
havior. However, behavioral descriptors are commonly used 
in research to operationalize, validate, communicate clearly 
among various groups, and assess program effectiveness. 
The developers of the MBI:TAC caution against an overly 
prescriptive list of specific mindful teaching behaviors, al- 
though examples are provided in the manual (Crane et al. 
2016). Mindfulness, by definition, requires negotiation of the 
“tricky territory of daily events as they unfold in a classroom” 
(Brown et al. 2016, p. 209). Thus, scale developers should 
attempt a balance of retaining the essence of foundational 
attitudes as they strive for constructing clear observational 
code definitions. Crane advises raters to maintain awareness 
of their own preferences and preconceptions about teaching 
style because “there is no single way to look embodied” 
(Crane et al. 2016, p. 8) or to embody mindful presence. 

With these considerations in mind, we adapted the basic 
MBI:TAC structure to construct a teacher rating scale 
(TMEOS) with a specific focus on the assessment of how well 
high school health teachers implemented the Learning to 
BREATHE (L2B) curriculum. Learning to BREATHE (L2B) 
is a mindfulness-based universal prevention program for ado- 
lescents that was designed to be integrated into educational 
settings and be compatible with school curricula. Learning to 
BREATHE is designed to help cultivate emotion regulation, 
attention, and performance, teach stress management, and help 
students integrate mindfulness into everyday life (see 
Broderick and Frank 2014; Broderick and Jennings 2012; 
Broderick and Metz 2016 for reviews). The program includes 
instruction in the practice of mindful awareness and provides 
opportunities to practice these skills in a group setting. L2B 
objectives are explicitly linked to standards for health, counsel- 
ing, and other professional areas so that the L2B program may 
be incorporated into existing curricula and assessment plans. 
Thus, the program was designed for classroom implementation 
and alignment with secondary school standards, and the goal 
of this study was to begin to develop a systematic way to assess 
the quality of such in-school implementations on the part of 
regular health education teachers. 

Six themes, built around the “BREATHE” acronym, form 
the core of the curriculum which may be adapted for various 
student groups and may be presented in six, 12, or 18 sessions. 
Each lesson includes a short introduction to the topic, a choice 
of activities for group participation and discussion to engage 
students in the lesson, and an opportunity for in-class mindful- 
ness practice. The core practices of mindfulness-based stress 
reduction (MBSR), including the body scan, awareness of 
thoughts and feelings, mindful movement, and loving kindness 
practices, are adapted for adolescents in the program. In these 
implementations, health teachers taught an orientation to stress 
lesson followed by 12 class sessions using selected activities, 
experiences, and practices in the L2B manual. 
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The current study describes the development and psycho- 
metric evaluation of an observational instrument, the Teaching 
Mindfulness in Education Observation Scale (TMEOS), 
which can be used to assess teacher implementation quality 
when delivering MBIs to youth. A central goal of the rating 
scale is to take into account unique factors of presenting MBIs 
ina school setting and to use language that fits the educational 
context. Although the instrument was developed for a specific 
MBI, Learning to BREATHE (Broderick 2013), we believe 
that it can be useful for those who offer other MBIs in educa- 
tional settings. 


Method 
Participants 


The present study took place in the context of a randomized 
control trial of Learning to BREATHE (L2B), a manualized 
mindfulness-based program for adolescents, by public high 
school teachers. Given the absence of measures to assess imple- 
mentation quality of MBI in education, we developed a new 
instrument to assess the “implementation quality” of the L2B 
program as utilized by high school teachers. Three high school 
health teachers were observed and videotaped teaching numer- 
ous sections of their classes over a 2-year period. Teachers taught 
the program during their required 11th grade health education 
class. The various iterations of the L2B program were videotaped 
and used for the development and validation of the TMEOS. The 
videos were drawn from three classes of 11th grade students in 
two large suburban high schools in the northeast USA that were 
composed of approximately 50% majority (Caucasian) and 50% 
ethnic minority (mostly African American) students. Consent to 
participate was obtained from teachers, students, and parents in 
accordance with university Institutional Review Board proce- 
dures and school district policies. 


Procedure 
Teacher Training 


L2B instructors varied somewhat in their degree of prior per- 
sonal mindfulness practice. Teacher | had an existing personal 
practice and had already attended a previous L2B training as 
well as a mindfulness retreat. Teachers 2 and 3 did not have 
prior experience with mindfulness practice nor with L2B. Prior 
to training in L2B, the program developer, on-site coach, and 
two teachers met after school for a four-session introduction to 
mindfulness program modeled on themes of MBSR. The ses- 
sions included weekly home practice and journaling to pro- 
mote a personal practice. All three teachers participated in an 
“intensive” training specifically focused on implementing 
L2B in their classrooms. The 2-day intensive training involved 
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practice teaching each lesson segment in small groups, sharing 
feedback with the whole group, and discussing practical details 
about implementation. In addition to the curriculum manual, 
teachers were given other materials, such as a “quick reference 
guide” to lesson structure and expanded outlines for sessions, 
developed to help them recognize key transitions in each les- 
son and to balance their use of time between activities, discus- 
sion, and mindfulness practice. One and/or both trainers visited 
teachers’ classrooms on a weekly or bi-weekly basis to observe 
classes and to provide feedback for improvement. Videotapes 
of each session were also made. Both trainers reviewed video- 
tapes for each session theme and had an average of four to six 
phone consultations with teachers over the course of each 12 
lesson program, in addition to the in vivo observations. Thus, 
teachers were introduced to personal mindfulness practice, par- 
ticipated in an intensive training on implementing L2B, were 
offered specialized program materials, and received on-site 
and phone coaching support during implementation. 
Teachers were free to contact the coaches at any time during 
program implementations. 


Measure Development 


Teachers taught L2B to at least five classes of students per 
semester. One teacher taught L2B during three separate se- 
mesters, and one teacher taught it over two semesters. A third 
teacher taught L2B over the course of one semester but was 
not included in the reliability analysis. Over the course of scale 
development, 80 class videotapes were observed. 

Scale domains were originally taken from the MBI:TAC 
(Crane et al. 2016), but, over time, key features were adapted 
to reflect the reality of teaching in actual secondary school 
classrooms. For example, the MBI:TAC description of 
patience (“The teaching process simply works as it is right 
now and allows an understanding that things can emerge in 
their own time”) was modified to include the pedagogical 
concept of wait-time as a means to encourage patience in 
classroom teachers. Early videotapes and observations (7 = 
68) were used in an iterative process to both refine the coding 
system as well as to provide exemplars that could be used as 
clear anchors for scaling. 


TMEOS Assessment Domains 


The TMEOS is comprised of four general domains including 
(1) planning, organization, and curriculum coverage; (2) teach- 
ing mindfulness; (3) guiding mindfulness practices; and (4) 
management of the learning environment. Domain 1 contains 
ten key features, and Domains 24 each contain six. While 
many of the key features from Domain | and 4 are typically 
used in assessing effective classroom functioning, Domains 2 
and 3 focus on unique elements involved in teaching an MBI in 
a school-type setting (see Supplementary materials online xxx). 
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Domain 1: Planning, Organization, and Curriculum Coverage 
Domain | assesses conventional aspects of teaching typically 
found in classrooms regardless of content presented. Key fea- 
tures include advanced preparation of materials, provision ofa 
conducive setting, effective use of class time, coverage and 
sequence of the elements of the lesson, clarity of instruction, 
accuracy of presentation, use of transitions to scaffold and link 
lesson elements, and use of clear and developmentally appro- 
priate examples. 


Domain 2: Teaching Mindfulness Domain 2 translates the 
foundational attitudes of mindfulness into teacher-friendly 
language using classroom examples. The first feature of 
Domain 2 addresses how teachers model the attitudes of cu- 
riosity and nonjudgment because teaching mindfulness re- 
quires a shift from top-down, content-heavy instruction 
(Pedagogy of Knowledge) to a more facilitative position 
(Pedagogy of Curiosity). Feature 2 addresses teachers’ aware- 
ness of present-moment experience in the classroom and abil- 
ity to maintain focus on present experience during the les- 
sons. The third feature assesses the quality of inquiry utilized 
during the lesson (i.e., questioning is directed toward noticing 
and exploring feelings and sensations and their patterns). 
Feature 4 assesses the balance of teacher talk versus student 
talk during lessons as well as class interactions. A fifth feature 
assesses teachers’ patience with the process of cultivating 
mindful awareness. Finally, feature 6 addresses the teacher’s 
willingness to be accepting of uncertainty and unpleasant 
experiences without reactivity. Domain 2 was particularly 
challenging to assess, and organization of key features went 
through multiple iterations. Developers relied both on videos 
and live observations of L2B classroom teaching to identify 
instances that demonstrated mindful attitudes in practice or, in 
some cases, the absence of the attitude. Overall, the ultimate 
goal was to identify, using authentic examples, how embodi- 
ment of mindfulness might “look” in the context of a regular 
classroom. 


Domain 3: Guiding Mindfulness Practices Domain 3 focuses 
on teachers’ guidance of mindfulness awareness practices dur- 
ing L2B lessons. The first feature of Domain 3 addresses how 
teachers create conditions conducive for practice, including 
the manner in which s/he settles the class and offers directions 
concerning posture. Feature 2 focuses on the vocal tone of the 
teacher during mindful practices. The third and fourth features 
assess the content (e.g., clear and accurate) and pace (e.g., 
smooth, appropriate pauses) of practices presented. Feature 5 
addresses how the teacher helps students better understand 
and engage in key attentional and attitudinal processes of each 
specific mindfulness practice. Finally, the sixth feature focuses 
on how teachers conclude each practice, including allowing 
sufficient time for individual students to return attention to the 
large group. 


Domain 4: Management of the Learning Environment Similar 
to Domain 1, Domain 4 includes assessment of effective 
teacher behaviors typically found in classroom settings. Key 
features of this domain include creating a safe and effective 
learning environment (e.g., use of group guidelines, confi- 
dentiality when sharing), using appropriate strategies to facil- 
itate student participation and engagement during lessons, 
fostering participation from a wide range of students, effec- 
tively addressing challenging student behaviors, motivating 
disengaged students, and attending to individual needs of 
students. 


Scoring Procedures 


For each domain, observers rate instructors on each of key 
features within the domain. Scores range from | to 3, 
using the anchor descriptions provided for each key fea- 
ture (see supplementary materials online xxx). For exam- 
ple, a key feature of Domain 1 (planning, organization and 
curriculum coverage) is a clear instruction for activities of 
the lesson. Raters score teachers’ performance on a con- 
tinuum from 1 (nstructions for activities are unclear or 
confusing to students) to 3 Unstructions are clear and 
succinct in accordance with the curriculum). A score of 
2 reflects general competence in the key feature with some 
minor exceptions (Teacher demonstrates minor lack of 
clarity or ease when giving instructions but this does not 
detract from the intent. Most instructions are clear.) More 
detailed descriptions of key features are available in the 
supplementary materials online (xxx). 

Domain proficiency scoring is based on an adaptation of 
the Dreyfus model utilized by Crane (Crane et al. 2013; 
Dreyfus and Dreyfus 1986). In the absence of prior data to 
guide cut-point selections, we created equal interval cut- 
points to characterize proficiency. Teachers receive a rating 
of “beginner” (average scores of 1—1.6) for a particular 
domain if inadequacy in performance is noted on most or 
all of the key features (i.e., scores of 1), thus implying the 
need for further skill development as well as possible close 
supervision. Teachers who receive average scores in the 
range of 1.7—2.3 may earn a rating of “competent.” There 
may be a few inconsistencies in the performance of 
teachers rated as competent, but their teaching reflects def- 
inite skill in the domain overall. A domain score between 
2.4 and 3 is considered “proficient.” Essentially, 
“proficient” teachers demonstrate skillful confidence and 
flexibility in lesson delivery while maintaining the integri- 
ty of the program. These teachers present lessons in a 
smooth, flexible, and responsive manner, indicating a 
deep, implicit understanding of the program. Skills are 
consistently demonstrated, even when difficulties arise 
(e.g., challenges from students, time or space constraints, 
interruptions within the school). 
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Observational Coding Process 


To assess the reliability and internal consistency of the 
TMEOS, 23 videotapes from the final implementation were 
coded by seven pairs of coders. Coders included a mixture of 
individuals including experts, who had an average of 13 years 
of experience in delivering mindfulness instruction and had 
served as lead instructors on MBIs on at least ten occasions, 
and novices who had not taught programs themselves but who 
had some experience in a mind-body practice like yoga or 
meditation. Training consisted of a whole group didactic train- 
ing session followed by coaching practice. Specifically, each 
coder received a manual containing the TMEOS coding pro- 
cedures, operational definitions of all codes, example sand 
non-examples, and step-by-step observational recording pro- 
cedures. During the training, coders watched a series of train- 
ing videos, which included demonstrations of coded behaviors. 
All coders worked independently during the coding process. 


Results 


Previous research suggests that the observational assessment of 
low base-rate behaviors may cause problems in the estimation 
of reliability (see Cicchetti et al. 1991). Therefore, prior to 
estimating inter-rater reliability, data were screened for any 
items with low-base rates or restricted range. Table | provides 
an overview of the mean, standard deviation, and range of 
instrument items and scales. The observability of behaviors 
was adequate across all assessed items and dimensions (see 
Tables | and 2). All domains displayed the full range of possi- 
ble scores and adequate variance. All analyses were conducted 
in IBM SPSS Statistics for Windows Version 23. 


Reliability and Specificity of Domain Scores 


Findings related to overall scale psychometrics for each do- 
main including internal consistency, average absolute agree- 
ment, and one-way random intra-class correlation coefficients 
(ICCs) are provided in Table 2. Overall, the four domains 
exhibited excellent internal consistency (range = 0.91—0.96) 
and strong ICC values ranging from 0.59 to 0.66. Absolute 
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coder agreement ranged from 70.08 to 82.99%; the average 
coder agreement across scales was 76.07%. These slightly 
lower values are not unexpected given this is a 3-point vs. 
dichotomous observational scale. The average item-total cor- 
relation for the planning, organization, and curriculum 
coverage domain was 0.81, the teaching mindfulness domain 
was 0.77, guiding mindfulness practices domain was 0.75, 
and management of the learning environment domain was 
0.80, all indicating very strong discrimination value among 
items comprising each domain. Pearson correlations were 
computed to examine the extent to which the four domains 
of the TMEOS represented related but distinct dimensions of 
behavior related to mindfulness instruction (see Table 4). 
Associations among domain scores were all positively corre- 
lated with a magnitude in the moderate to high range (range = 
0.72—0.89) in expected directions (see Tables 3 and 4). 


Discussion 


This paper describes the development of a reliable rating sys- 
tem for assessing fidelity in classroom teachers’ implementa- 
tion of MBIs in school settings. The Teaching Mindfulness in 
Education Observation Scale is intended to assess both the 
structural (adherence) and process (quality) aspects of pro- 
gram implementation. The scale was designed to be used with 
classroom teachers and other educators using the L2B pro- 
gram for youth, primarily in school or out-of-school contexts. 
While some aspects of the rating system are naturally specific 
to the program components of L2B, we believe that this sys- 
tem could be effectively used with slight modifications with 
other MBI-ED programs. 

Similar to other social and emotional learning programs, it 
was important that the TMEOS incorporated quantitative and 
qualitative aspects of implementation, capturing both adher- 
ence and implementation quality (Dobkin et al. 2014; Gould 
et al. 2014). Furthermore, following from the work of Crane 
et al. (2013), the TMEOS carefully assesses the teacher’s em- 
bodiment of foundational attitudes of mindfulness. Not surpris- 
ingly, the results show that behavioral aspects of program ad- 
herence were somewhat easier to assess reliably than aspects of 
implementation quality like teacher embodiment. Given the 


Table 1 Domain level descriptive statistics 

Item Scale 
Domain Mean Standard deviation Range Mean Standard deviation Range 
Planning, organization, and curriculum coverage 2.10 0.61 1-3 21.00 6.15 11-30 
Teaching mindfulness 1.67 0.60 1-3 10.02 3:57 6-18 
Guiding practices 1.78 0.55 1-3 10.70 3.33 6-18 
Learning environment 1.92 0.62 1-3 11.53 3.72 6-18 
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Table 2. Domain item level agreement 


Mean Range 


Domain 1: Planning, organization and curriculum coverage 


1.1. Physical environment 2.17 1-3 
1.2. Preparation 2.33 1-3 
1.3. Coverage 2.22 1-3 
1.4. Time usage 2.07 1-3 
1.5. Information accuracy 2.00 1-3 
1.6. Order of lesson 1.98 1-3 
1.7. Clarity of instructions 1.96 1-3 
1.8. Transitions 1.93 1-3 
1.9. Appropriate examples 2.24 1-3 
1.10. Avoid extraneous content 2.11 1-3 


Domain 2: Teaching mindfulness 


2.1. Pedagogy of curiosity 1.61 1-3 
2.2. Focus on present moment 1.74 1-3 
2.3. Inquiry directed 1.70 1-3 
2.4. Student talk time 1.37 1-3 
2.5. Teacher patience 1.85 1-3 
2.6. Teacher acceptance 1.76 1-3 
Domain 3: Guiding mindfulness practices 

3.1. Optimal practice conditions 2.00 1-3 
3.2. Vocal tone 1.80 1-3 
3.3. Clarity of guidance 1.74 1-3 
3.4 Pacing of guidance 159 1-3 
3.5. Effective guidance 1.78 1-3 
3.6. Appropriate duration 1.80 1-3 
Domain 4: Management of the learning environment 

4.1. Safe and effective environment 1.89 1-3 
4.2. Engagement strategies 1.78 1-3 
4.3. Encourages participation 2.07 1-3 
4.4. Behavior management 1.97 1-3 
4.5. Effective motivation 1.90 1-3 
4.6. Effective differentiation 2.05 1-3 


essential nature of embodiment for effective teaching of mind- 
fulness, it seems clear that, in both training and assessment 
efforts for classroom teachers, cultivating teachers’ own mind- 
fulness practice on a sustained basis is essential (see Roeser 
2016). We also concur with Crane that it is important for 


Table 3. Domain scale internal consistencies 


Standard deviation Item-total correlation Absolute agreement 
0.68 0.61 57.89% 
0.70 0.80 68.42% 
0.73 0.84 84.21% 
0.65 0.86 73.68% 
0.84 0.88 78.95% 
0.75 0.84 73.68% 
0.79 0.83 78.95% 
0.77 0.83 89.47% 
0.57 0.76 73.68% 
0.77 0.85 89.47% 
0.68 0.86 78.95% 
0.71 0.80 73.68% 
0.79 0.84 68.42% 
0.57 0.57 73.68% 
0.67 0.77 63.16% 
0.79 0.80 63.16% 
0.79 0.87 89.47% 
0.69 0.71 78.95% 
0.53 0.81 94.74% 
0.54 0.68 89.47% 
0.63 0.80 84.21% 
0.79 0.65 61.11% 
0.80 0.73 42.11% 
0.73 0.75 73.68% 
0.77 0.87 84.21% 
0.77 0.83 69.23% 
0.75 0.85 83.33% 
0.57 0.75 90.00% 


evaluators to have prior mindfulness and program experience 
before rating others’ teaching any mindfulness program. As 
stated earlier, there is not one right way to demonstrate embod- 
ied mindfulness. The essence of the foundational attitudes 
should be understood both conceptually and procedurally so 


Domain Cronbach’s alpha Average absolute agreement Intraclass correlation 
Estimate CI 
Planning, organization, and curriculum coverage 0.96 76.84% 0.66 0.56-0.77 
Teaching mindfulness 0.92 70.18% 0.62 0.51-0.74 
Guiding practices 0.91 82.99% 0.59 0.47-0.71 
Learning environment 0.93 73.76% 0.66 0.54-0.78 
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correlations Domain 1 2 3 4 
1. Planning, organization, and curriculum coverage 1 
2. Teaching mindfulness 0.72** 1 
3. Guiding practices 0.89** 0.74** 1 
4. Management of learning environment 0.79** 0.87** 0.72** 1 


**Indicates p value less than 0.01 


that they can be recognized as they are flexibly, authentically, 
and variously manifested in real classrooms. 

It should be reiterated that behaviors and attitudes are not 
mutually exclusive entities, and most of the key features do 
contain both behavioral and attitudinal elements. However, 
some key features are more behavioral and concrete than 
others, and these primarily tended to reflect program adher- 
ence. For example, Domain 1: planning, organization, and 
curriculum coverage contained key features, such as lesson 
structure, that are specific to activities and practices covered in 
the lesson. Both percent agreement and inter-observer reliabil- 
ity assessed with actual scores (ICC) for this domain were 
relatively high. Although within the acceptable range, 
Domain 2 and 3: teaching mindfulness and guiding practices 
showed comparatively weaker inter-coder agreement and ICC 
values (see Table 2). These domains capture manifestations of 
the attitudinal foundations of mindfulness and strategies for 
creating conditions conducive to mindfulness practices, which 
may require a deep understanding of mindfulness and, per- 
haps, somewhat greater subjectivity compared to the other 
three domains. Domain 4: management of the learning 
environment showed adequate agreement and ICC values; 
however, it is important to note that not all key features from 
this domain can necessarily be evidenced in every class ob- 
served. For example, two key features addressed the teacher’s 
ability to handle noncompliant and unmotivated students ef- 
fectively while maintaining group cohesion, which may or 
may not be present in all cases. In the majority of classes 
observed in this study, students were largely cooperative, so 
this feature was sometimes scored as “not-applicable.” 
However, the TMEOS is relatively flexible with regard to 
what constitutes “student misbehavior,” and individual inves- 
tigators may impose more specific requirements regarding 
what constitutes an appropriate behavioral response. Our goal 
was to produce a generalizable instrument, but teachers and 
researchers adopting the TMEOS may choose to implement 
more specific criteria for their population or context regarding 
student behavior and appropriate teacher responses. 

Despite the potential applicability of this scale, there are 
several limitations that should be addressed in further studies. 
This scale was developed using observations from a limited 
number of teachers. This allowed for a highly in-depth under- 
standing of teacher development as these individuals 
transitioned from a more directive teaching style to a more 
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facilitative, experiential one to accommodate the pedagogy 
of the program. Working intensely with a small number of 
teachers over time allowed us to observe and describe 
teachers’ “emerging embodiment of mindfulness” at close 
range. However, a broader array of teachers, including those 
with more experience in teaching and practicing mindfulness, 
could reveal other patterns of strengths and difficulties that 
might be incorporated into this scale. Therefore, evaluating 
this instrument with larger and more diverse teacher samples 
and settings (e.g., special education) could strengthen its 
usefulness. 

It would be important to also explore numeric cut-points 
used to characterize levels of proficiency. Although beyond 
the scope of the present paper, examining the meaningfulness 
of proficiency discriminations is an important area for future 
research. We used raters to assess teacher videos who repre- 
sented a range of mindfulness and/or L2B experience them- 
selves. Future work should carefully examine the contributions 
that personal mindfulness practice and program experience of 
the assessors contribute to the overall reliability of the instru- 
ment. Finally, future research using these domains may be able 
to assess the relative contributions of program adherence and 
quality of implementation to overall program effectiveness. 

Despite these limitations, the above evidence suggests that 
the TMEOS might be useful not only as an observational scale 
for evaluative purposes, but also as a tool for scaffolding the 
teacher coaching/learning process. Instructor development is a 
critical component of effective MBI programming, particular- 
ly in the area of embodying mindfulness. Because MBIs by 
definition are practice-intensive, teacher mindfulness provides 
the essential procedural model whereby students learn how to 
be mindful themselves. No mindfulness curriculum alone can 
accomplish its intended goals. The teacher’s capacity to im- 
plement the curriculum effectively rests on both his/her grasp 
of the program as well as embodiment of mindfulness as a 
way of being. We believe that the TMEOS could be used in 
the training process as a rubric for teachers’ self-assessments 
of behaviors and attitudes as they develop expertise. 

On the one hand, the high inter-correlations we observed 
between the domain scores suggest that a total “Quality of 
Implementation Score” may be usefully derived from this 
measure for general purposes. On the other hand, with larger 
samples of teachers who had different capacities and back- 
ground experience in mindfulness, we might expect more 
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differentiation in the domain scores and thereby, greater spec- 
ificity around targets for classroom teacher support. Finally, 
our work on this paper suggests a need for TMEOS assessors 
to embody mindfulness and be well-versed in the behavioral 
and attitudinal aspects of the curriculum being implemented in 
preparation for assessing others’ level of expertise. 

The issue of teacher quality and embodiment of mindfulness 
remains a relatively unexplored issue with regard to teaching 
mindfulness to children and youth. This study represents an 
attempt to bridge the gap between what is considered essential 
to the teaching of adult mindfulness programs with the reality of 
teaching mindfulness in educational settings. Use of a relevant 
observational instrument in combination with best-practice ex- 
amples and supportive feedback may better equip teachers to 
shift from the “doing” mode of highly task-oriented systems to 
the kind of “being-mode” that supports mindful awareness. 
Therefore, a psychometrically validated instrument that accu- 
rately reflects teachers’ expertise in program adherence and 
embodiment of mindfulness may be a useful component in 
facilitating their development and can make the teaching of 
mindfulness more accessible and effective in schools. 
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